ja: handle furigana — two kinds, opposite treatment (closes #704) - #705
Merged
Conversation
Before: a parenthesised reading was phonemized as ordinary text after the kanji it annotates, so the reading came out TWICE — `漢字(かんじ)` read *känd͡ʑi känd͡ʑi* where a reader says it once. The issue proposed one rule. Implementing it turned up a distinction worth splitting on: TRUE RUBY POSITIONING IS NOT PLAIN TEXT, so what reaches a phonemizer is one of several flattenings of markup — and they do not all mean the same thing. DECLARED — Unicode interlinear annotation (U+FFF9 base U+FFFA ruby U+FFFB, defined for exactly this and discouraged for interchange, so rare) and the aozora `|base《ruby》`. These SAY they are ruby, so the author has stated the reading: the RUBY WINS and the base is dropped. `|日本《にっぽん》` now reads にっぽん, overriding the default にほん — which is the entire reason for writing it. PARENTHESISED — `base(ruby)` / `base(ruby)`. A CONVENTION, not a declaration; the same shape is an ordinary parenthetical. So the guard is EQUALITY with the computed reading, and a mismatch means "keep both" rather than "the author is overriding". The three cases the issue named as must-not-change all survive, because none equals the computed reading: 日本(にほんじん) a gloss 日本(にっぽん) an alternate reading in the AMBIGUOUS form 会議(ミーティング) a katakana gloss of a loanword Note 日本(にっぽん) and |日本《にっぽん》 now differ, and that is the point: the parenthesised one is a parenthetical the author may have meant as a contrast, the declared one is an instruction. The test asserts the property that actually matters — an annotated sentence is BYTE-IDENTICAL to the same sentence written without the annotation, including at the token boundary the rewrite creates. That caught the one thing worth checking: dropping the annotation joins 東京 directly to に, and the result must equal `東京に行く`. It does. 3114 → 3118 tests; typecheck clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012Lc3WnUgogC7okV7n53vjr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #704.
Before: a parenthesised reading was phonemized as ordinary text after the kanji it annotates, so the reading came out twice —
漢字(かんじ)readkänd͡ʑi känd͡ʑi, where a reader says it once.The issue proposed one rule; implementing it turned up a distinction worth splitting on
True ruby positioning is not plain text. What reaches a phonemizer is one of several flattenings of markup — and they don't all mean the same thing.
DECLARED — Unicode interlinear annotation (
U+FFF9 base U+FFFA ruby U+FFFB, defined for exactly this and discouraged for interchange, so rare) and the aozora|base《ruby》. These say they are ruby, so the author has stated the reading: the ruby wins and the base is dropped.That's the entire reason for writing it.
PARENTHESISED —
base(ruby)/base(ruby). A convention, not a declaration; the same shape is an ordinary parenthetical. So the guard is equality with the computed reading, and a mismatch means "keep both" rather than "the author is overriding".The must-not-change cases all survive
None equals the computed reading, so none is touched:
日本(にほんじん)niho̞ꜜɴ niho̞nd͡ʑiɴ日本(にっぽん)niho̞ꜜɴ nippo̞ꜜɴ会議(ミーティング)käꜜiɡi miːtiŋɡɯᵝ⚠ Note
日本(にっぽん)and|日本《にっぽん》now differ, and that is the point: the parenthesised one is a parenthetical the author may have meant as a contrast; the declared one is an instruction.The test asserts the property that actually matters
Not "the output looks right" but: an annotated sentence is byte-identical to the same sentence written without the annotation — including at the token boundary the rewrite creates.
That caught the one thing worth checking. Dropping the annotation joins 東京 directly to に, so the result has to equal
東京に行く:Checks
npm run typecheckclean🤖 Generated with Claude Code
https://claude.ai/code/session_012Lc3WnUgogC7okV7n53vjr